CONVERGENCE TO STABLE LAWS 
IN RELATIVE ENTROPY 

S. G. BOBKOV^^*, G. P. CHISTYAKOV^-'', AND F. GOTZE'*''* 

Abstract. Convergence to stable laws in relative entropy is established for sums of i.i.d. 
random variables. 



1. Introduction 

Given independent identically distributed random variables {Xn)n>i, consider the normal- 
ized sums 

y _ Xi + ■ ■ ■ + Xn _ 

On 

defined for given (non-random) a„ E R and 6„ > 0. Assume that Z„ converges weakly in 
distribution to a random variable Z which has a non-degenerate stable law. In this paper, we 
would like to study whether or not this convergence holds in a stronger sense. This question has 
been studied and affirmatively solved in the literature, for example, using the total variation 
distance between the distributions of Z„ and Z (cf. [I-L]). Results in the central limit theorem 
suggest to consider for instance the stronger "entropic" distance, that is, the relative entropy, 
where, however, not so much is known. Similarly one might consider the convergence in terms 
of the closely related Fisher information (a question raised in [J], Ch.5). Convergence in such 
distances may be viewed as part of a theoretic-information approach to limit theorems, which 
has been initiated by Linnik [Li], who studied the behavior of the entropy of sums of regularized 
independent summands in the central limit theorem. 

Given random variables X and Z with distributions // and v, respectively, the relative 
entropy of (j. with respect to z^, sometimes called informational divergence or Kullback-Leibler 
distance, of /x and is defined by 



D{X\\Z) = D{ii\\v) = j log^df,, 



provided that fi is absolutely continuous with respect to (and otherwise D(X||Z) = +00). 

We consider the relative entropy with respect to so-called non-extremal stable laws (cf. 
relations (1.1) below and the definition before them). The aim of this note is to prove: 
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Theorem 1.1. Assume that the sequence of normalized sums defined above converges 
weakly to a random variable Z with a non- extremal stable limit law. Then the relative entropy 
distances converge to zero, that is, D(Zn\\Z) — >■ 0, as n — >■ oo, if and only if D{Zn\\Z) < +oo, 
for some n. 

In the sequel, we consider non-degenerate distributions, only. 

If Xi has a finite second moment, weak convergence Z^ ^ Z holds with a„ = -y/n EXi, 
by^ = ^yn, and Z being normal. In this case. Theorem 1.1 turns into the entropic central 
limit theorem by Barron [B]; cf. also [B-C-G] for refinements and a different approach. Thus, 
Theorem 1.1 may be viewed as an extension of Barron's result. 

If Xi has an infinite second moment, but still belongs to the domain of normal attraction, 
it follows that D{Zn\\Z) = +oo for all n. Hence, in this special case there is no convergence 
in relative entropy. 

In the remaining cases Z has a stable distribution with some parameters < a < 2 and 
— 1 < /3 < 1. It then has a continuous density ip{x) with characteristic function f{t) = Ee**^ 
described by the formula 

log f{t) = exp [iat — c|t|"(l + i/3sign(t) uj{t, a)) }, 

where a G R, c > 0, and u{t, a) = tan(^) in case a ^ 1, and uj{t, a) = ^ log \t\ for a = 1. In 
particular, = e~'^l*l". 

Exact expressions for the characteristic function are, however, not sufficiently informative 
for establishing results like Theorem 1.1. In order to pass from weak convergence to a stronger 
convergence, we need more information about the stable densities ip. 

This information in turn will depend on the type of a given stable distribution. A stable 
distribution is called non-extrem,a,l, if it is normal or, if < a < 2 and —1 < (5 < 1. In 
the latter case, the density of Z is positive on the whole real line and satisfies asymptotic 
relations 

i){x) ~ Co {x -oo), i:{x) ~ ci a;-(i+") {x +00) (1.1) 

with some constants co,ci > (cf. [I-L], [Z]). 

The behavior of if) in the extremal case (when = 1) is different. For example, when 
< a < 1, the density is positive on a half-axis H = (.xq, +00) or H = (— oo,xo) of the real 
line, only. Hence, to guarantee finiteness of the relative entropies D(Zn\\Z), one has to require 
that Zn take values in H (which involves a certain requirement on the coefficients o„ and bn)- 
Another important issue is that, as x — )■ xq, ip{x) — >■ extremely fast, so the finiteness of 
D{Zn\\Z) leads as well to an additional strong moment assumption about the distribution of 
Xi near a point. A similar effect may be observed in the case 1 < a < 2, = 1, as well. Here 
is positive everywhere, but tends to zero extremely fast either near +00 or —00 (especially, 
when a = 1). 

Note that the property that Xi belongs to the domain of attraction of a stable law of index 
< q; < 2 may be expressed explicitly in terms of the distribution function F{x) = P{Xi < x}. 
Namely, we have Zn ^ Z with some bn > and a„ G R, if and only if 

F{x) = (co + o(l))|xr°S(|x|) (x^-oo), 
1-F{x) = {ci + o{l))x-°' B{x) (x^+00). 
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for some constants co,ci > that are not both zero, and where B{x) is a slowly varying 
function in the sense of Karamata (cf. [I-L], [Z]). 

Furthermore, in the non-cxtrcmal case the condition D{Zn\\Z) < +00 in Theorem 1.1 is 
equivalent to saying that has a density Pn with finite entropy 

/+00 
Pn{x) log Pn{x)dx 
-00 

(with an additional requirement that EX^ < +00, when Z is normal). Once this property 
is fulfilled for a particular value n = no, it continues to hold for all n > uq. This will be 
explained in the next section. 

We shall turn to the proof which we divide into several steps. For completeness, the 
argument will cover the normal case as well. 



2. Remarks on Relative Entropy 

First, let us give necessary and sufficient conditions for the property D[Zn\\Z) < +00, 
especially when Z has a non-extremal stable distribution. 

Given a random variable X with density p, consider the entropy functional 

h(X) = — j p{x) log p{x)dx. 

In general, it may or may not be defined as a Lebcsgue integral. (Here and below, we often 
omit the limits of integration when integrating over the whole real line.) 

It is well-known that, if X has a finite second moment, then h{X) is well-defined, and one 
has an upper estimate 

h{X) < h{Z), (2.1) 

where Z is & normal random variable with the same mean and variance as X. (Here the value 
h{X) = —00 is possible.) This important observation may be generalized with respect to other 
(not necessarily normal) reference measures. 



Proposition 2.1. Let X denote a random variable with density p. Assume that ^ denotes 
a probability density on the real line, such that ^{x) = implies p{x) = a.e., and such that 
E log"^ ( ^^-^^ ) < +00. Then h{X) exists and satisfies 



M^)<Elog-^. (2.2) 



This bound seems to be folklor knowledge and is based on a direct application of Jensen's 
inequality. To recall the argument, let Z have density ■0, and assume that tp{x) = implies 
p{x) = a.e. The relative entropy given by 

D{X\\Z)= [p{x)log^dx = EClogC, ? = ^, 

is then well-defined, where the expectation refers to the probability space (R, ijj{x) dx). More- 
over, since E^ = 1, and due to the convexity of the function t — >■ tlogt, the expectation 
E^log^ exists and is non-negative. But plog^ +plog| = plog:^ and, by the assumption. 
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J plog ^ dx exists and does not take the value +00. Hence, h{X) = J plog ^ dx also exists and 
cannot take the value +00. In addition, the equality 

D{X\\Z) = -h{X) + Blog^ (2.3) 

is justified. Here, the left-hand side is non-negative, so the inequality (2.2) immediately follows. 

Choosing for ip a normal density with the same mean and the same variance as for X, one 
may easily see that (2.2) reduces to (2.1). 

As another example, choosing for the density of the Cauchy measure, we obtain a weaker 
moment condition Elog(l-|- |X|) < -|-oo, which guarantees that the entropy exists and satisfies 
h{X) < +00. 

Let us return to the definition of the relative entropy. 



DiX\\Z) = Jp{x)log^^dx, 



assuming that Z has density V- In order to describe when this distance is finite, one may 
complement Proposition 2.1 with the following: 

Proposition 2.2. Let X be a random variable with density p. Assume that 

a) ip{x) = =^ p{x) = a.e.; 

Elog+(^) < +00; 
c) h{X) is finite. 

Then D{X\\Z) is finite and is given by (2.3). Conversely, if J ip{x)'^ dx < +00, for some 
< 7 < 1, then the conditions a) — c) are also necessary for the relative entropy to be finite. 

Proof. Condition a) means that the distribution of X is absolutely continuous with respect 
to the distribution of Z (which is necessary for the finiteness of the relative entropy). 
Assuming condition a), we have D{X\\Z) < +00, if and only if the integral 



is finite. Let / indeed be finite. Split the real line into the two sets A = {x : p{x) > ip{x)'^'} 
and B = 'R\A, where 7 < 7' < 1. Restricting the integration to A, we get 



I 



> j^p{x)Xog{l + ,^)dx > (1-7') /^P(x) log+(^)^- 



Hence, p{x) log"'"(;^^) dx < +00. On the other hand, using iT log(l/t) < CiT (0 < t < 1), 
we get 

/ p{x) \og-^ {-)—) dx < I tp^xy' \og+ {--^) dx 
Jb 'V(a;) Jb Xx)' 

= / i^i^V' log (tTt) ^{V'W<i} dx < C tp{xy dx < +00. 

JB J-00 
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As a result, / p{x) log'''(;^;^) dx < +00, that is, b) is fulfilled. 

But then, by Proposition 2.1, the entropy is well-defined in the Lebesgue sense and, more- 
over, h{X) < +00. This justifies writing (2.3), which implies that h{X) > —00 as well. Hence, 
the property c) holds as well. 

In the other direction, assuming that a) — c) are fulfilled, one may start with equality (2.3), 
which shows that L'(X||Z) is finite. 

Thus, Proposition 2.2 is proved. 

When ijj is the density of a stable law, the condition J iIj{x)^ dx < +00 (for some < 7 < 1) 
is fulfilled, so the properties a) — c) are necessary and sufficient for the finiteness of the relative 
entropy with respect to ip. In fact, a more detailed conclusion may be stated according to the 
types of stable laws. 

Corollary 2.3. If Z is normal, then D{X\\Z) < +00, if and only if X has a finite second 
moment and finite entropy. 

Corollary 2.4. // Z has a non-extremal stable distribution, which is not normal, then 
D{X\\Z) < +00, if and only if X has a finite logarithmic moment Elog(l + \X\) and finite 
entropy. 

This follows from Proposition 2.2 and the property (1.1). Let us recall that the condition 
Elog(l + \X\) < +00 insures that the entropy of X exists and, moreover, h{X) < +00. 

The situation where Z has an extremal stable distribution is a bit more delicate, but may 
be studied on the basis of Proposition 2.2 as well. However, we do not discuss this case here. 

These characteristions may be simplified for normalized sums Z„ = ^1+^"+^" _ with 
i.i.d. summands as in Theorem 1.1, provided that the sequence Z„ is weakly convergent in 
distribution. Indeed, the property Zn =^ Z, where Z has a stable distribution with parameter 
< a < 2 implies that E \Xi\^ < +00 for any < s < a, cf. [I-L], [Z]. Hence, E \Zn\^ < +00 
for all n > 1, and thus the random variables Zn have finite logarithmic moments. 

Corollary 2.5. Assume that Z„ converges weakly to a random variable Z with a non- 
extremal stable limit law, which is not normal. Then, for each n > 1, the finiteness of the 
relative entropy D{Zn\\Z) is equivalent to the finiteness of the entropy of Z^. 

A similar conclusion holds in the normal case as well, provided that 6„ ~ y/n. Here it is 
well-known that Zn ^ Z implies that E < +00. 

Finally, let us mention another property of non-extremal stable distributions. 

Corollary 2.6. Assume that Z has a non-extremal stable distribution. If the relative 
entropy D{Zn\\Z) is finite for some n = no, it will be finite for all n> uq. 

Proof. By Jensen's inequality, h{X + Y) > h{X) for all independent summands such 
that h{X) exists and h{X) > —00. If D{ZnQ\\Z) < +00, then h{Zno) is finite according to 
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Proposition 2.2. Hence, for the sums Sn = Xi + • • • + Xn, h{Sn) exists for all n > uq and 

h{Sn) > h{Sno) = log6„o + h{Zn„) > -oo. 

Thus, h{Zn) exists with h{Zn) > — oo. In addition, by Corollary 2.4 (if Z is not normal), 
we also have that E log(l + l-^no I) < +oo. By convexity of the function u — )■ log(l + u) (?x > 0), 
this yields Elog(l + l-'^il) < +oo. In turn, since log(l + u + v) < log(l + u) + log(l + v) 
{u,v > 0), we get Elog(l + \Sn\) < +oo, for all n. In particular, h{Sn) < +oo, according to 
Proposition 2.1. Therefore, h{Zn) is finite, and applying Proposition 2.2, we conclude that 
D{Zn\\Z) is finite for all n > tiq. 

Using Corollary 2.3, a similar argument applies to the normal case as well. 

Note that Corollary 2.6 does not extend to the class of extremal stable distributions. 



3. Binomial Decomposition of Convolutions 

Given independent identically distributed random variables {Xn)n>i and numbers a„ G R, 
6„ > 0, consider the sums 

g 

Sn = Xi^ \-Xn and Z„ = - a„. 

On 

If 2^ is a random variable with an absolutely continuous distribution (not necessarily stable) , 
the condition D{Zn\\Z) < +oo (n > jiq) used in Theorem 1.1 implies that, for any such n, 
Zn has an absolutely continuous distributions with density, say Pn{x)- For simplicity, wc may 
and will assume that uq = 1, that is, already Xi has a density p{x). (The case, where Zn have 
densities starting from n > no with uq > 1 requires minor modifications only). 

Since it is advantageous in the following to work with bounded densities, we slightly modify 
Pn at the expense of a small change in the relative entropy. For a given number < 6 < ^, 
split H into the two Borel sets Hi and Hq, such that p is bounded by a constant M on Hi 
with 



b = p{x) dx. 
Jho 

Consider the decomposition 

p{x) = {l-b)pi{x) + bpoix), (3.1) 

where pi, po are the normalized restrictions of p to the sets Hi and Hq, respectively. Hence, 
for the convolutions we have a binomial decomposition 



p*^ = ^ (1 - 5)^ b""-^ pf * p*^"-'^) 

fe=0 



This function represents the density of Sn- 

For n > 2, we split the above sum into the two parts, so that p*"' = pni + PnO with 



p„i = 2_^C„ (1 - 6) b pi * Pq' PnO = Po +n{l-b)b Pi*Pq 

k=2 
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Note that 

e, 

Finally define 



J pnoix) dx = h^ + n{l- b)h^-^ < n6"-\ (3.2) 



Pn{x) = ^" pnl {an + Kx) , Pno{x) = — Pno{an + bnX). (3.3) 

Thus, for the densities p„ of Z„ we have the decomposition 

Pn{x) = (1 - en)Pn{x) + EnPnoix). (3.4) 

The (probability) densities pn are bounded and provide a strong approximation for pn 
regardless of the choice of numbers a„ and 6„ from the definition of Z^. In particular, from 
(3.2) and (3.4) and using 6 < ^, it follows that 



\pn{x)-pn{x)\dx<2-'', (3.5) 

for all n large enough. One of the immediate consequences of this estimate is the bound 

|/n(t)-/„(t)| <2-" (tGR) (3.6) 
for the corresponding characteristic functions 

fnit) = J e'*^p„(x)dx, fnit) = J e^*^pn(x)dx. 

Under mild conditions on a„ and bn, the approximation (3.5) may be sharpened by using 
a polynomial weight function in the L^-distance. 

Lemma 3.1. //E|Xi|* < +00 (s > 0), and \an\ + 1/bn = 0{rC) with some 7 > 0, then 
for all n large enough, 



J \x\'\pn{x)-pn{x)\dx <2 



Proof. We refine arguments from the proof of a similar Lemma 2.1 in [B-C-G]. By (3.4), 

\Pn{x) -Pn{x)\ < en{Pn{x) +Pno{x)), 

SO 

J \x\^ \Pnix) - Pn{x)\ dx < b~^ j \x - Pnl {x) dx 

+ b~^ J \x - anl" Pnoix)dx. (3.7) 

Let Ui,U2, ■ ■ ■ be independent copies of U and Vi,V2, ■ ■ ■ be independent copies of V (that 
are also independent of all Un), where U and V are random variables with densities pi and po, 
respectively. Prom (3.2) 

Ps = 'E\Xi\' = (l-6)E|^7|^ + 6E|F|^ (3.8) 
so E \U\^ < Ps/b and E \V\^ < Ps/b (using b < ^). Consider the sums 

Sk,n = Ui + --- + Uk + Vi + --- + Vn-k, 0<k<n. 
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If s > 1, then by the triangle inequaUty in the space with norm \\^\\s = (E|4|'^)^/'^, we get 

k n 

\\Sk,n\\s < + E ll^^-ll- = k\\U\\s + (.n-k)\\V\U < n(/3,/6)V^ (3.9) 

j=l j=k+l 

Hence, E l/Sfc^^l* < and, by Jensen's inequahty, E \Sk,n — o-n]^ < 2*(^ + lonH- 

If < s < 1, one can just use 

\Sk,n\' < |C/ir + • • • + \Uk\' + + • • • + \Vn-k\', 

implying that E l-Sfe^^l* < nfig/h and E \Sk,n — 0'n\^ + \anf ■ In both cases, 

E \Sk,n - < 2^ n--(^'i) + |a„r^ . 

Hence, 

/n 
\x-an\'pnl{x)dx = Y,C!^{l-b)H^-''E\Sk,n-an\' 
k=2 



s 



[\x-an\'pno{x)dx = ^ (1 - 6)^"-*^ E - a, 
The two estimates may be used in (3.7), and we get 

J \x\'\pn{x)-pn{x)\dx < ^(^^n--(^'l) + Kr^£„. 

It remains to apply (3.2) together with the assumption on {an,bn)- Lemma 3.1 is proved. 
Lemma 3.2. For any to > 0, there are positive constants c and C such that, for all n>2, 

[ |/n(t)|dt < C5ne-^". 

J\t\>tob„ 

Proof. Consider the densities p = pf' * p*Q^~^^ appearing in the definition of p„i. Their 
Fourier transforms (i.e., the corresponding characteristic functions) are connected by 

p(i) = /3i(t)^PoW"-*^ (teR). 

By the Riemann-Lebesgue theorem, \pj{t)\ < e"'^, for all \t\ > to with some constant c > 
{j = 0, 1). Hence, whenever 2 < k < n. 

|p(^)|<A|pl(^)|^ A = e-<^-'\ 
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By the decomposition construction, pi{x) < M, for all x. Applying Plancherel's formula, we 
get 

/ \p{t)\dt<A / \pi{t)\'^ dt = 277 A / pi{xfdx<2'iTAM. 

J\t\>to J J 

As a consequence, the density pni satisfies a similar inequality 



i\t\>u 

1^ " 



[ \pni{t)\dt <2TrAM{l-en). 

J\t\>tn 



But, by (3.3), fn{t) = j^e-'"'"/''-pni{t/bn), so 

[ \fnit)\ dt = [ \pnl{t)\ dt < 2'KMhn e'^''-''^ 

J\t\>tQbn J- — En J\t\>to 

Thus, Lemma 3.2 is proved. 

Remark 3.3. If Z„ ^ Z, where Z has a stable distribution of index < a < 2, then 
necessarily 5„ ~ •n}/^B{n), as n — t- oo, where S is a slowly varying function in the sense of 
Karamata (cf. [I-L]). Using standard arguments (cf. e.g. [La]), one can show as well that 
Qn = o{n). Thus, the conditions of Lemma 3.1 for the coefficients (a„,6„) are fulfilled, once 
there is a weak convergence. 

4. Entropic Approximation of by p„ 

We need to extend the assertion of Lemma 3.1 to the relative entropies with respect to the 
stable laws. Thus, assume that Z has a stable distribution of index a G (0, 2] with density 
il){x) (thus including the normal law). Put 



Dn = D 



{Zn\\Z) = J p„(x)log^ dx, Dn = J p„(x)log^ dx, 



where Pn are defined according to the decomposition (3.4) for the densities Pn of the normalized 
sums 

Zn = —{Xi + ---+Xn)-an (On G R, bn > 0). 

Here, as before, denote independent identically distributed random variables. 

In the lemma below, it does not matter whether or not the sequence Z^ converges weakly 
to Z. 

Lemma 4.1. Assume that the distribution of Z is non-extremal. If is finite for all 
n > no, and |o„| + log6n + 1/bn = O^n"') with some 7 > 0, then 

ID — D \ < 2~" 

for all n large enough. 



Proof. To simplify the notations (and the argument), we assume that no = 1. In particular, 
Di = D{Xi\\Z) is finite, hence the entropy h{Xi) is finite and Elog ^{x^) < +co, according 
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to Proposition 2.2. Define 



DnO = / PnO{X) log . dx. 



By convexity of the function L{u) = nlogn {u > 0) it follows that Dn < {l — £n)Dn+£nDnO 
and, as easy to see, 

Dn > ((1 - £n)Dn + EnDno) + log + (1 " ^n) log(l - 

By the definition (3.2) of £„, the two estimates give 

\bn-Dr,\<Cn{n + bn + DnQ)lp-\ (4.1) 

which holds for all n > 1 with some constant C. In addition, using an elementary inequality 
L((l - b)u + hv)>{l - h) L(u) -\u-\ (it, > 0, < 6 < 1), we get from (3.1) that 

D{X^\\Z) = j L{^M^^{x)dx>{l-b) I p,(x)log^dx-l 

A similar inequality also holds for po with b in place of 1 — 6, so 

D{X,\\Z) > (1 - b)D{U\\Z) - ^, D{X,\\Z) > bD{V\\Z) - ^, 

where U and V have densities pi and po, respectively. Hence, these random variables have 
finite entropies, and by Proposition 2.2, 

Elog^<+oo, Elog^<+oo. (4.2) 

Let Ui,U2, ■ ■ ■ be independent copies of U and let Vi, V2, . . . be independent copies of V 
(which are independent of all C/„ as well). Again, by convexity of the function ulogu, 

^ T3i;E^n(l-&)'«'"-'/^M(^)log^^cia;, (4.3) 

DnO < -Yl (1 - b)' / r.,„(x) log dx, (4.4) 

where rk,n are the densities of the normalized sums 
S 1 

Rk,n = ^-an = — {Ui + --- + Uk + Vi + --- + Vn-k) -an, < k < n. 
On bn 

Now, write 

/^k nix) f 1 

rk,n{x) log ^^^^ dx = -h{Rk,n) + J rk,n{x) log dx, (4.5) 

using the entropy functional h{R) = — J r{x)logr{x) dx. Adding independent summands to 
R will only increase the value of this functional. Hence, for any 1 < A; < n, 

h{Rk,n) = -\ogbn + hiUi + --- + Uk + Vi + --- + Vn-k) 
> -\ogbn + h{U). 
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For k = there are similar relations (with V replacing U), so whenever < < n, we have 
h{Rk,n) > — login — C with some constant C. Inserting in (4.5), we arrive at 

D{Rk,n\\Z)<logbn + C + -Elog—^. (4.6) 

Case 1: Z ~ N{a,a^) with some a e R and cr > 0. Using (4.6) and (3.8)-(3.9) with s = 2 
and the assumption EX^ < +oo (due to the assumption Di < +oo, cf. Corollary 2.3), we get 

D{Rk,n\\Z) < l0gbn + Ci+C2E\Rk,n\^ 

V? o 

< \oghn + C^ + Ci -J + Czal 

with some constants Cj depending on a,a,b and EXf. Using the condition on and 6^2, 
we conclude that D{Rk^n\\Z) < with some 7' for all n large enough. Applying this in 
(4.3)-(4.4), (4.1) yields \Dn - Dn\ = o{Vl), whenever 6 < 61 < ^. 

Case 2: Z has a non-extremal stable distribution. By (4.2), we have Elog(l + \U\) < +00 
and similarly for V. In addition, by (1.1), 

Elog— ^ < A + fiElog(l + \Rk,n\) (4.7) 

V{Rk,n) 

with some constants A and B. To bound the last expectation, one may use the inequality 
log(l + l^ii +U2I) < log(H- |wi|) +log(l + \ u2\), valid for all real numbers ui, U2, together with 
log(l + u) < l/u + log u {u > 0). They yield 

log(l + \Rk,n\) < log(l + |a„|) + log(l + \Sk,„\/bn), 

while log(l + \Sk,n\/bn) < '^/K + log(l + \Sk,n\)- Prom this, we get 

k n 

iog(i + |i?jt,„|) < i + iog(i + K|) + ^iog(i + |c/,|)+ iog(i + |y,|), 

3=1 j=k+l 

and therefore 

Elog(l + |i?fe,„|) < -^ + log(l + |a„|) + A:Elog(l + |i7|) + (n-fe)Elog(l + |y|) 

< +log(l + |a„|) + Cn. 

Thus, by (4.7) and (4.6), with some constant C 

D{Rk,n\\Z) < C{n + log 6n + l/hn + log(l + |an|)) . 

It remains to apply this bound in (4.3)-(4.4), and then (4.1) yields |D„ — D„| = 0(6") with 
any 61 > 6. One may take b\ = \, and thus Lemma 4.1 is proved. 
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5. Uniform Local Limit Theorem 

Consider the normalized sums Zn = y- {Xi + • • • + X„) — a„ (where Un G R, bn > 0), 
associated to independent identically distributed random variables X^. 

Proposition 5.1. Assume that Z„ =^ Z, where Z has a (continuous) density ip. If the 
random variables Z^ have absolutely continuous distributions for n > tiq with densities, say 
Pn, then 

sup \pnix) — '4'{x)\ — )■ (n — )■ oo). (5-1) 

X 

Here Pn denote the modified densities of Pn, constructed in Section 3 for the case no = 1. 
Necessarily, Z has a stable distribution of some index a G (0, 2], and bn ~ n^^°'B{n), where B 
is a slowly varying function. 

Note that in this proposition it does not matter, whether Z is extremal or not. 

Proof. Consider the characteristic functions 

/+0O f+OO 
e'''^Pnix)dx, f{t)= / e''''4^{x)dx, 
-oo J — oo 

and express the densities via inverse Fourier transforms, while splitting the Fourier integral 
into the two regions, 

Pnix) - = ^ / e-^*- iUt) - fit)) dt + ^ [ e-''- Unit) - fit)) dt 

with given io > and < < to^n- 

Let /„ denote the characteristic functions of By assumption, if — >■ +oo sufiiciently 
slowly, then r„max|t|<j;^ - 0. Hence, by (3.6), 

Tn inax \Jnit) - fit)\ ^0 (n ^ oo), 

1^1 ^'^n 

SO that uniformly in all x 

Pnix) - V(x) = ^ / e-^*- iUt) - fit)) dt + o(l). 
2^ J\t\>T„ 

Moreover, since / is integrable, 

sup \pnix) - ^Pix)\ < ^ / \fnit)\ dt + o(l). (5.2) 
X 27r J\t\>T„ 

Recall that the characteristic functions are integrable as well. The integration in (5.2) 
should also be splitted into the two regions accordingly, and hence the integral itself will be 
bounded by 



/ \fnit)\dt+ f \fnit)\dt. 

JTn<\t\<tobn •J\t\>tobn 
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But, by Lemma 3.2, the last integral tends to zero, as long as bn has at most polynomial 
growth. Using once more (3.6), we arrive at 

sup \pn{x) - i;{x)\ <^ [ \fn{t)\ dt + o(l). 

It remains to apply the following bound derived in [I-L], p. 133. There exist constants c > 
and to > 0, such that \ fn{t)\ < e~'^l*l"'^^ for all n > 1 and t in the interval \t\ < tobn- This gives 

/ \fn{t)\dt< [ e-^l*l"^' ^ 0. 

JT„<\t\<tobn J\t\<tobn 

Thus, Proposition 5.1 is proved. 

6. Proof of Theorem 1.1 

Let X and Z be random variables with densities p and ip, such that ip{x) = p{x) = 
a.e. The relative entropy 

D{X\\Z) = P^logP^i^{x)dx (6.1) 

is well-defined and may be bounded from above by applying an elementary inequality 

tlogt < {t - 1) + Ce\t - (t>0), (6.2) 

where depends on e G (0, 1], only. Namely, it immediately yields 

Moreover, letting A = sup^. \p{x) — tp{x)\, 

/+0O 1 
-^\p{x)-^p{x)\dx. (6.3) 

This is a general upper bound which may be used in the proof of Theorem 1.1 in case of 
a non-normal stable density ^p of index < a < 2 and using X = Zn with modified densities 
Pn- Indeed, by Proposition 5.1, 

An = sup \pn{x) - 'il;{x)\ ^ (n^oo). 

X 

In addition, if Z is non-extremal, V' admits a lower bound tp{x) > c{l + \x\)~^^'^°'^ with some 
constant c > 0, cf. (1.1). Hence, by (6.3), 

/ + 00 
(1 + |x|)^(l+°) \pn{x)-4^{x)\dx, 
-oo 

where the constant depends on e and ip. But, for an arbitrary e < so that s = £(l+a) < a, 
we see that the last integral does not exceed 

J{l + \x\y \pn{x)-pn{x)\dx + J {1 + \x\Y Pnix) dx + J {1 + \x\y tp {x) dx . 
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All these integrals are bounded by a constant, which follows from Lemma 3.1 and the fact that 
sup„ E < +00 (which is due to the assumption Z„ Z, cf. [I-L]). As a result, we have 
D{Zn\\Z) — )• 0, which yields the desired conclusion D{Zn\\Z) — )• in view of Lemma 4.1. 

In the normal case (a = 2), a similar argument with slight modifications may be applied 
as well. Without loss of generality, assume that Z is standard normal, i.e., V'(^) = 
Now we use (6.2) with e = 1 and = 1. More precisely, splitting the integration in (6.1) into 
the two regions, we get 

D{X\\Z)<[ M^il^l#^dx+/ p(x)log^dx (6.4) 



l\t\<T i^{x) J\t\>T 

with an arbitrary T > 0. Furthermore, the first integral on the right-hand side is bounded by 



e^^/^V2^A [ \p{x) -'ilj{x)\dx <2e^^/^V2^A. 

J\t\<T 

If additionally p{x) < M, (6.4) leads to another general upper bound 

D{X\\Z) <2e^^/^V2^A+ I {x^ + log{MV2^)p{x) dx. (6.5) 

J\t\>T 

Here, again let X = Zn and T = Tn. Then the above bound holds with M = 1 and all n 
large enough. If T„ — ?• +00 sufficiently slow, we have e"^"/^A„ — )• 0. On the other hand, 

r+00 



X Pn{x)dx < I X \pn{x) - Pn{x)\ dx 
|t|>T„ J -00 

+ I x"^ pn{x) dx + I x'^ ilj{x) dx. 

J\t\>T„ J\t\>T„ 

Again, all the integrals tend to zero, in view of Lemma 3.1 and the uniform integrability of 
the sequence Z^. Hence, by (6.5), D{Zn\\Z) — )■ 0, which, by Lemma 4.1, proves Theorem 1.1 
in the normal case. 
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